LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data.

Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be 40× faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

Establishment of a Multiplex Detection Method for Common Bacteria in Blood Based on Human Mannan-Binding Lectin Protein-Conjugated Magnetic Bead Enrichment Combined with Recombinase-Aided PCR Technology.

Objective: Recombinase-aided polymerase chain reaction (RAP) is a sensitive, single-tube, two-stage nucleic acid amplification method. This study aimed to develop an assay that can be used for the early diagnosis of three types of bacteremia caused by Staphylococcus aureus (SA), Pseudomonas aeruginosa (PA), and Acinetobacter baumannii (AB) in the bloodstream based on recombinant human mannan-binding lectin protein (M1 protein)-conjugated magnetic bead (M1 bead) enrichment of pathogens combined with RAP. Methods: Recombinant plasmids were used to evaluate the assay sensitivity. Common blood influenza bacteria were used for the specific detection. Simulated and clinical plasma samples were enriched with M1 beads and then subjected to multiple recombinase-aided PCR (M-RAP) and quantitative PCR (qPCR) assays. Kappa analysis was used to evaluate the consistency between the two assays. Results: The M-RAP method had sensitivity rates of 1, 10, and 1 copies/µL for the detection of SA, PA, and AB plasmids, respectively, without cross-reaction to other bacterial species. The M-RAP assay obtained results for < 10 CFU/mL pathogens in the blood within 4 h, with higher sensitivity than qPCR. M-RAP and qPCR for SA, PA, and AB yielded Kappa values of 0.839, 0.815, and 0.856, respectively ( P < 0.05). Conclusion: An M-RAP assay for SA, PA, and AB in blood samples utilizing M1 bead enrichment has been developed and can be potentially used for the early detection of bacteremia.

Randomness Regularization with Simple Consistency Training for Neural Networks.

Randomness is widely introduced in neural network training to simplify model optimization or avoid the over-fitting problem. Among them, dropout and its variations in different aspects (e.g., data, model structure) are prevalent in regularizing the training of deep neural networks. Though effective and performing well, the randomness introduced by these dropout-based methods causes nonnegligible inconsistency between training and inference. In this paper, we introduce a simple consistency training strategy to regularize such randomness, namely R-Drop, which forces two output distributions sampled by each type of randomness to be consistent. Specifically, R-Drop minimizes the bidirectional KL-divergence between two output distributions produced by dropout-based randomness for each training sample. Theoretical analysis reveals that R-Drop can reduce the above inconsistency by reducing the inconsistency among the sampled sub structures and bridging the gap between the loss calculated by the full model and sub structures. Experiments on 7 widely-used deep learning tasks ( 23 datasets in total) demonstrate that R-Drop is universally effective for different types of neural networks (i.e., feed-forward, recurrent, and graph neural networks) and different learning paradigms (supervised, parameter-efficient, and semi-supervised). In particular, it achieves state-of-the-art performances with the vanilla Transformer model on WMT14 English → German translation ( 30.91 BLEU) and WMT14 English → French translation ( 43.95 BLEU), even surpassing models trained with extra large-scale data and expert-designed advanced variants of Transformer models. Our code is available at GitHub https://github.com/dropreg/R-Drop.

NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality.

Text-to-speech (TTS) has made rapid progress in both academia and industry in recent years. Some questions naturally arise that whether a TTS system can achieve human-level quality, how to define/judge that quality, and how to achieve it. In this paper, we answer these questions by first defining the human-level quality based on the statistical significance of subjective measure and introducing appropriate guidelines to judge it, and then developing a TTS system called NaturalSpeech that achieves human-level quality on benchmark datasets. Specifically, we leverage a variational auto-encoder (VAE) for end-to-end text-to-waveform generation, with several key modules to enhance the capacity of the prior from text and reduce the complexity of the posterior from speech, including phoneme pre-training, differentiable duration modeling, bidirectional prior/posterior modeling, and a memory mechanism in VAE. Experimental evaluations on the popular LJSpeech dataset show that our proposed NaturalSpeech achieves -0.01 CMOS (comparative mean opinion score) to human recordings at the sentence level, with Wilcoxon signed rank test at p-level p >> 0.05, which demonstrates no statistically significant difference from human recordings for the first time.

Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing.

Geometric deep learning has been revolutionizing the molecular modeling field. Despite the state-of-the-art neural network models are approaching ab initio accuracy for molecular property prediction, their applications, such as drug discovery and molecular dynamics (MD) simulation, have been hindered by insufficient utilization of geometric information and high computational costs. Here we propose an equivariant geometry-enhanced graph neural network called ViSNet, which elegantly extracts geometric features and efficiently models molecular structures with low computational costs. Our proposed ViSNet outperforms state-of-the-art approaches on multiple MD benchmarks, including MD17, revised MD17 and MD22, and achieves excellent chemical property prediction on QM9 and Molecule3D datasets. Furthermore, through a series of simulations and case studies, ViSNet can efficiently explore the conformational space and provide reasonable interpretability to map geometric representations to molecular structures.

Multi-omic analysis revealed the therapeutic mechanisms of Alpinia oxyphylla fructus water extract against bladder overactivity in spontaneously hypertensive rats.

OBJECTIVE: Alpinia oxyphylla fructus without impurities and shells is called "Yi-Zhi-Ren" (YZR) in Chinese, and traditionally used to alleviate enuresis. The aim of this study was to investigate the effects and underlying mechanisms of YZR in the treatment of overactive bladder (OAB) in spontaneously hypertensive rats (SHR), a vascular disorder-related OAB model. METHODS: A 3-week administration of YZR water extract (p.o.) was done, followed by urodynamics to measure bladder parameters. Changes in bladder structure were observed through H&E staining and Masson's staining. An integrated approach involving network pharmacology, transcriptomics and metabolomics was employed to elucidate the potential mechanisms of YZR, and the key proteins involved in the mechanisms were validated by Western blotting. Additionally, network pharmacology was used to predict the relationship between YZR's active components and validated proteins. RESULTS: YZR treatment significantly improved the bladder storage parameters, tightened the detrusor layer, reduced inflammatory infiltration, and decreased collagen proportion in the SHR bladder. These results indicated that YZR water extract can alleviate OAB symptoms and improve bladder structure. Integrated analysis suggested that YZR may affect extracellular matrix-receptor interaction and calcium signaling pathway. Western blotting results further confirmed that the reduction in key proteins, such as TGFß1, p-SMAD3, collagen III, Gq and PLCß1, involved in collagen synthesis and calcium signaling pathways after YZR treatment. Network pharmacology predicted that sitosterol, chrysin, and nootkatone were potential components responsible for YZR's therapeutic effect on OAB. CONCLUSION: YZR's mechanisms of action in treating OAB involved the TGFß1-SMAD3 signaling pathway-related collagen synthesis and Gq-PLCß1 calcium signaling pathway, which are associated with detrusor contraction frequency and strength, respectively.

Multi-omic approaches provide insights into the molecular mechanisms of Sojae semen germinatum water extract against overactive bladder.

Sojae semen germinatum (SSG) is derived from mature soybean seeds that have been germinated and dried, typically with sprouts measuring approximately 0.5 cm in length. SSG is traditionally known for its properties in clearing heat and moisture. Nevertheless, limited information was reported on the effects and mechanisms of SSG in alleviating urinary symptoms. This study employed urodynamic parameters to investigate the therapeutic effect of SSG water extract on overactive bladder (OAB) in the rat model with benign prostatic hyperplasia. Through a combination of transcriptomic and metabolomic analyses, the pathways and key proteins of the SSG treatment for OAB were identified and validated by ELISA and Western blotting. Furthermore, network pharmacology elucidated the roles of SSG's isoflavones acting on the target which was identified by above-mentioned multi-omics analysis. Our results indicate that SSG water extract significantly mitigated OAB by down-regulating the PGE2/EP1/PLCß2/p-MLC signaling pathway. It was speculated that the active ingredient in the SSG on EP1 was genistein. This study provided valuable insights into the molecular mechanisms of SSG water extract, emphasizing the multi-target characteristics and critical pathways in improving OAB. Furthermore, this study contributes to the potential utilization of SSG as a functional food.

Breaking the barriers of data scarcity in drug-target affinity prediction.

Accurate prediction of drug-target affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities. While wet experiments remain the most reliable method, they are time-consuming and resource-intensive, resulting in limited data availability that poses challenges for deep learning approaches. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. To overcome this challenge, we present the Semi-Supervised Multi-task training (SSM) framework for DTA prediction, which incorporates three simple yet highly effective strategies: (1) A multi-task training approach that combines DTA prediction with masked language modeling using paired drug-target data. (2) A semi-supervised training method that leverages large-scale unpaired molecules and proteins to enhance drug and target representations. This approach differs from previous methods that only employed molecules or proteins in pre-training. (3) The integration of a lightweight cross-attention module to improve the interaction between drugs and targets, further enhancing prediction accuracy. Through extensive experiments on benchmark datasets such as BindingDB, DAVIS and KIBA, we demonstrate the superior performance of our framework. Additionally, we conduct case studies on specific drug-target binding activities, virtual screening experiments, drug feature visualizations and real-world applications, all of which showcase the significant potential of our work. In conclusion, our proposed SSM-DTA framework addresses the data limitation challenge in DTA prediction and yields promising results, paving the way for more efficient and accurate drug discovery processes.

Complex-valued neural-operator-assisted soliton identification.

The numerical determination of solitary states is an important topic for such research areas as Bose-Einstein condensates, nonlinear optics, plasma physics, and so on. In this paper, we propose a data-driven approach for identifying solitons based on dynamical solutions of real-time differential equations. Our approach combines a machine-learning architecture called the complex-valued neural operator (CNO) with an energy-restricted gradient optimization. The CNO serves as a generalization of the traditional neural operator to the complex domain, and constructs a smooth mapping between the initial and final states; the energy-restricted optimization facilitates the search for solitons by constraining the energy space. We concretely demonstrate this approach on the quasi-one-dimensional Bose-Einstein condensate with homogeneous and inhomogeneous nonlinearities. Our work offers an idea for data-driven effective modeling and studies of solitary waves in nonlinear physical systems.

Scientific discovery in the age of artificial intelligence.

Artificial intelligence (AI) is being increasingly integrated into scientific discovery to augment and accelerate research, helping scientists to generate hypotheses, design experiments, collect and interpret large datasets, and gain insights that might not have been possible using traditional scientific methods alone. Here we examine breakthroughs over the past decade that include self-supervised learning, which allows models to be trained on vast amounts of unlabelled data, and geometric deep learning, which leverages knowledge about the structure of scientific data to enhance model accuracy and efficiency. Generative AI methods can create designs, such as small-molecule drugs and proteins, by analysing diverse data modalities, including images and sequences. We discuss how these methods can help scientists throughout the scientific process and the central issues that remain despite such advances. Both developers and users of AI toolsneed a better understanding of when such approaches need improvement, and challenges posed by poor data quality and stewardship remain. These issues cut across scientific disciplines and require developing foundational algorithmic approaches that can contribute to scientific understanding or acquire it autonomously, making them critical areas of focus for AI innovation.

Surgical Treatment of Osteosarcoma Induced Distant Pre-Metastatic Niche in Lung to Facilitate the Colonization of Circulating Tumor Cells.

Recently, the major challenge in treating osteosarcoma patients is the metastatic disease, most commonly in the lungs. However, the underlying mechanism of recurrence and metastasis of osteosarcoma after surgical resection of primary tumor remains unclear. This study aims to investigate whether the pulmonary metastases characteristic of osteosarcoma is associated with surgical treatment and whether surgery contributes to the formation of pre-metastatic niche in the distant lung tissue. In the current study, the authors observe the presence of circulating tumor cells in patients undergoing surgical resection of osteosarcoma which is correlated to tumor recurrence. The pulmonary infiltrations of neutrophils and Gr-1+ myeloid cells are characterized to form a pre-metastatic niche upon the exposure of circulating tumor cells after surgical resection. It is found that mitochondrial damage-associated molecular patterns released from surgical resection contribute to the formation of pre-metastatic niche in lung through IL-1ß secretion. This study reveals that surgical management for osteosarcoma, irrespective of the primary tumor, might promote the formation of postoperative pre-metastatic niche in lung which is with important implications for developing rational therapies during peri-operative period.

AIMD-Chig: Exploring the conformational space of a 166-atom protein Chignolin with ab initio molecular dynamics.

Molecular dynamics (MD) simulations have revolutionized the modeling of biomolecular conformations and provided unprecedented insight into molecular interactions. Due to the prohibitive computational overheads of ab initio simulation for large biomolecules, dynamic modeling for proteins is generally constrained on force field with molecular mechanics, which suffers from low accuracy as well as ignores the electronic effects. Here, we report AIMD-Chig, an MD dataset including 2 million conformations of 166-atom protein Chignolin sampled at the density functional theory (DFT) level with 7,763,146 CPU hours. 10,000 conformations were initialized covering the whole conformational space of Chignolin, including folded, unfolded, and metastable states. Ab initio simulations were driven by M06-2X/6-31 G* with a Berendsen thermostat at 340 K. We reported coordinates, energies, and forces for each conformation. AIMD-Chig brings the DFT level conformational space exploration from small organic molecules to real-world proteins. It can serve as the benchmark for developing machine learning potentials for proteins and facilitate the exploration of protein dynamics with ab initio accuracy.

Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics.

Machine learning force fields (MLFFs) have gained popularity in recent years as they provide a cost-effective alternative to ab initio molecular dynamics (MD) simulations. Despite a small error on the test set, MLFFs inherently suffer from generalization and robustness issues during MD simulations. To alleviate these issues, we propose global force metrics and fine-grained metrics from element and conformation aspects to systematically measure MLFFs for every atom and every conformation of molecules. We selected three state-of-the-art MLFFs (ET, NequIP, and ViSNet) and comprehensively evaluated on aspirin, Ac-Ala3-NHMe, and Chignolin MD datasets with the number of atoms ranging from 21 to 166. Driven by the trained MLFFs on these molecules, we performed MD simulations from different initial conformations, analyzed the relationship between the force metrics and the stability of simulation trajectories, and investigated the reason for collapsed simulations. Finally, the performance of MLFFs and the stability of MD simulations can be further improved guided by the proposed force metrics for model training, specifically training MLFF models with these force metrics as loss functions, fine-tuning by reweighting samples in the original dataset, and continued training by recruiting additional unexplored data.

A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond.

Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and natural language processing communities. While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation. In recent years, many new models and algorithms have been designed/proposed to bridge the accuracy gap between NAR generation and AR generation. In this paper, we conduct a systematic survey with comparisons and discussions of various non-autoregressive translation (NAT) models from different aspects. Specifically, we categorize the efforts of NAT into several groups, including data manipulation, modeling methods, training criterion, decoding algorithms, and the benefit from pre-trained models. Furthermore, we briefly review other applications of NAR models beyond machine translation, such as grammatical error correction, text summarization, text style transfer, dialogue, semantic parsing, automatic speech recognition, and so on. In addition, we also discuss potential directions for future exploration, including releasing the dependency of KD, reasonable training objectives, pre-training for NAR, and wider applications, etc. We hope this survey can help researchers capture the latest progress in NAR generation, inspire the design of advanced NAR models and algorithms, and enable industry practitioners to choose appropriate solutions for their applications.

Hybrid aortic arch procedure in acute type A aortic dissection with right carotid artery occlusion.

Acute type A aortic dissection complicated by carotid artery is associated with a high risk of perioperative stroke. We reported a case of application of hybrid aortic arch debranching procedure in acute type A aortic dissection complicated by right carotid artery occlusion, which resulted in no neurological complications after operation and patent carotid artery after discharging.

DSN-DDI: an accurate and generalized framework for drug-drug interaction prediction by dual-view representation learning.

Drug-drug interaction (DDI) prediction identifies interactions of drug combinations in which the adverse side effects caused by the physicochemical incompatibility have attracted much attention. Previous studies usually model drug information from single or dual views of the whole drug molecules but ignore the detailed interactions among atoms, which leads to incomplete and noisy information and limits the accuracy of DDI prediction. In this work, we propose a novel dual-view drug representation learning network for DDI prediction ('DSN-DDI'), which employs local and global representation learning modules iteratively and learns drug substructures from the single drug ('intra-view') and the drug pair ('inter-view') simultaneously. Comprehensive evaluations demonstrate that DSN-DDI significantly improved performance on DDI prediction for the existing drugs by achieving a relatively improved accuracy of 13.01% and an over 99% accuracy under the transductive setting. More importantly, DSN-DDI achieves a relatively improved accuracy of 7.07% to unseen drugs and shows the usefulness for real-world DDI applications. Finally, DSN-DDI exhibits good transferability on synergistic drug combination prediction and thus can serve as a generalized framework in the drug discovery field.

Targeted therapy for head and neck cancer: signaling pathways and clinical studies.

Head and neck cancer (HNC) is malignant, genetically complex and difficult to treat and is the sixth most frequent cancer, with tobacco, alcohol and human papillomavirus being major risk factors. Based on epigenetic data, HNC is remarkably heterogeneous, and treatment remains challenging. There is a lack of significant improvement in survival and quality of life in patients with HNC. Over half of HNC patients experience locoregional recurrence or distal metastasis despite the current multiple traditional therapeutic strategies and immunotherapy. In addition, resistance to chemotherapy, radiotherapy and some targeted therapies is common. Therefore, it is urgent to explore more effective and tolerable targeted therapies to improve the clinical outcomes of HNC patients. Recent targeted therapy studies have focused on identifying promising biomarkers and developing more effective targeted therapies. A well understanding of the pathogenesis of HNC contributes to learning more about its inner association, which provides novel insight into the development of small molecule inhibitors. In this review, we summarized the vital signaling pathways and discussed the current potential therapeutic targets against critical molecules in HNC, as well as presenting preclinical animal models and ongoing or completed clinical studies about targeted therapy, which may contribute to a more favorable prognosis of HNC. Targeted therapy in combination with other therapies and its limitations were also discussed.

R2-DDI: relation-aware feature refinement for drug-drug interaction prediction.

Precisely predicting the drug-drug interaction (DDI) is an important application and host research topic in drug discovery, especially for avoiding the adverse effect when using drug combination treatment for patients. Nowadays, machine learning and deep learning methods have achieved great success in DDI prediction. However, we notice that most of the works ignore the importance of the relation type when building the DDI prediction models. In this work, we propose a novel R$^2$-DDI framework, which introduces a relation-aware feature refinement module for drug representation learning. The relation feature is integrated into drug representation and refined in the framework. With the refinement features, we also incorporate the consistency training method to regularize the multi-branch predictions for better generalization. Through extensive experiments and studies, we demonstrate our R$^2$-DDI approach can significantly improve the DDI prediction performance over multiple real-world datasets and settings, and our method shows better generalization ability with the help of the feature refinement design.